54 research outputs found
Data Mining the SDSS SkyServer Database
An earlier paper (Szalay et. al. "Designing and Mining MultiTerabyte
Astronomy Archives: The Sloan Digital Sky Survey," ACM SIGMOD 2000) described
the Sloan Digital Sky Survey's (SDSS) data management needs by defining twenty
database queries and twelve data visualization tasks that a good data
management system should support. We built a database and interfaces to support
both the query load and also a website for ad-hoc access. This paper reports on
the database design, describes the data loading pipeline, and reports on the
query implementation and performance. The queries typically translated to a
single SQL statement. Most queries run in less than 20 seconds, allowing
scientists to interactively explore the database. This paper is an in-depth
tour of those queries. Readers should first have studied the companion overview
paper Szalay et. al. "The SDSS SkyServer, Public Access to the Sloan Digital
Sky Server Data" ACM SIGMOND 2002.Comment: 40 pages, Original source is at
http://research.microsoft.com/~gray/Papers/MSR_TR_O2_01_20_queries.do
The Sloan Digital Sky Survey Science Archive: Migrating a Multi-Terabyte Astronomical Archive from Object to Relational DBMS
The Sloan Digital Sky Survey Science Archive is the first in a series of
multi-Terabyte digital archives in Astronomy and other data-intensive sciences.
To facilitate data mining in the SDSS archive, we adapted a commercial database
engine and built specialized tools on top of it. Originally we chose an
object-oriented database management system due to its data organization
capabilities, platform independence, query performance and conceptual fit to
the data. However, after using the object database for the first couple of
years of the project, it soon began to fall short in terms of its query support
and data mining performance. This was as much due to the inability of the
database vendor to respond our demands for features and bug fixes as it was due
to their failure to keep up with the rapid improvements in hardware
performance, particularly faster RAID disk systems. In the end, we were forced
to abandon the object database and migrate our data to a relational database.
We describe below the technical issues that we faced with the object database
and how and why we migrated to relational technology
GridCertLib: a Single Sign-on Solution for Grid Web Applications and Portals
This paper describes the design and implementation of GridCertLib, a Java
library leveraging a Shibboleth-based authentication infrastructure and the
SLCS online certificate signing service, to provide short-lived X.509
certificates and Grid proxies. The main use case envisioned for GridCertLib, is
to provide seamless and secure access to Grid/X.509 certificates and proxies in
web applications and portals: when a user logs in to the portal using
Shibboleth authentication, GridCertLib can automatically obtain a Grid/X.509
certificate from the SLCS service and generate a VOMS proxy from it. We give an
overview of the architecture of GridCertLib and briefly describe its
programming model. Its application to some deployment scenarios is outlined, as
well as a report on practical experience integrating GridCertLib into portals
for Bioinformatics and Computational Chemistry applications, based on the
popular P-GRADE and Django softwares.Comment: 18 pages, 1 figure; final manuscript accepted for publication by the
"Journal of Grid Computing
The SDSS SkyServer, Public Access to the Sloan Digital Sky Server Data
The SkyServer provides Internet access to the public Sloan Digital Sky Survey
(SDSS) data for both astronomers and for science education. This paper
describes the SkyServer goals and architecture. It also describes our
experience operating the SkyServer on the Internet. The SDSS data is public and
well-documented so it makes a good test platform for research on database
algorithms and performance.Comment: submitted for publication, original at
http://research.microsoft.com/scripts/pubs/view.asp?TR_ID=MSR-TR-2001-10
High-Redshift Quasars Found in Sloan Digital Sky Survey Commissioning Data IV: Luminosity Function from the Fall Equatorial Stripe Sampl
This is the fourth paper in a series aimed at finding high-redshift quasars
from five-color imaging data taken along the Celestial Equator by the SDSS.
during its commissioning phase. In this paper, we use the color-selected sample
of 39 luminous high-redshift quasars presented in Paper III to derive the
evolution of the quasar luminosity function over the range of 3.6<z<5.0, and
-27.5<M_1450<-25.5 (Omega=1, H_0=50 km s^-1 Mpc^-1). We use the selection
function derived in Paper III to correct for sample incompleteness. The
luminosity function is estimated using three different methods: (1) the 1/V_a
estimator; (2) a maximum likelihood solution, assuming that the density of
quasars depends exponentially on redshift and as a power law in luminosity and
(3) Lynden-Bell's non-parametric C^- estimator. All three methods give
consistent results. The luminous quasar density decreases by a factor of ~ 6
from z=3.5 to z=5.0, consistent with the decline seen from several previous
optical surveys at z<4.5. The luminosity function follows psi(L) ~ L^{-2.5} for
z~4 at the bright end, significantly flatter than the bright end luminosity
function psi(L) \propto L^{-3.5} found in previous studies for z<3, suggesting
that the shape of the quasar luminosity function evolves with redshift as well,
and that the quasar evolution from z=2 to 5 cannot be described as pure
luminosity evolution. Possible selection biases and the effect of dust
extinction on the redshift evolution of the quasar density are also discussed.Comment: AJ accepted, with minor change
Photometric Redshifts of Quasars
We demonstrate that the design of the Sloan Digital Sky Survey (SDSS) filter
system and the quality of the SDSS imaging data are sufficient for determining
accurate and precise photometric redshifts (``photo-z''s) of quasars. Using a
sample of 2625 quasars, we show that photo-z determination is even possible for
z<=2.2 despite the lack of a strong continuum break that robust photo-z
techniques normally require. We find that, using our empirical method on our
sample of objects known to be quasars, approximately 70% of the photometric
redshifts are correct to within delta z = 0.2; the fraction of correct
photometric redshifts is even better for z>3. The accuracy of quasar
photometric redshifts does not appear to be dependent upon magnitude to nearly
21st magnitude in i'. Careful calibration of the color-redshift relation to
21st magnitude may allow for the discovery of on the order of 10^6 quasars
candidates in addition to the 10^5 quasars that the SDSS will confirm
spectroscopically. We discuss the efficient selection of quasar candidates from
imaging data for use with the photometric redshift technique and the potential
scientific uses of a large sample of quasar candidates with photometric
redshifts.Comment: 29 pages, 8 figures, submitted to A
KL Estimation of the Power Spectrum Parameters from the Angular Distribution of Galaxies in Early SDSS Data
We present measurements of parameters of the 3-dimensional power spectrum of
galaxy clustering from 222 square degrees of early imaging data in the Sloan
Digital Sky Survey. The projected galaxy distribution on the sky is expanded
over a set of Karhunen-Loeve eigenfunctions, which optimize the signal-to-noise
ratio in our analysis. A maximum likelihood analysis is used to estimate
parameters that set the shape and amplitude of the 3-dimensional power
spectrum. Our best estimates are Gamma=0.188 +/- 0.04 and sigma_8L = 0.915 +/-
0.06 (statistical errors only), for a flat Universe with a cosmological
constant. We demonstrate that our measurements contain signal from scales at or
beyond the peak of the 3D power spectrum. We discuss how the results scale with
systematic uncertainties, like the radial selection function. We find that the
central values satisfy the analytically estimated scaling relation. We have
also explored the effects of evolutionary corrections, various truncations of
the KL basis, seeing, sample size and limiting magnitude. We find that the
impact of most of these uncertainties stay within the 2-sigma uncertainties of
our fiducial result.Comment: Fig 1 postscript problem correcte
High-Redshift Quasars Found in Sloan Digital Sky Survey Commissioning Data VI. Sloan Digital Sky Survey Spectrograph Observations
We present results on over 100 high-redshift quasars found in the Sloan
Digital Sky Survey (SDSS), using automated selection algorithms applied to SDSS
imaging data and with spectroscopic confirmation obtained during routine
spectroscopic operations of the Sloan 2.5-m telescope. The SDSS spectra cover
the wavelength range 3900--9200 Angstroms at a spectral resolution of 1800, and
have been obtained for 116 quasars with redshifts greater than 3.94; 92 of
these objects were previously uncataloged, significantly increasing the current
tally of published z>4 quasars. The paper also reports observations of five
additional new z>4.6 quasars; all were found from the SDSS imaging survey and
spectroscopically confirmed with data from the Apache Point Observatory's 3.5-m
telescope. The i' magnitudes of the quasars range from 18.03 to 20.56. Of the
97 new objects in this paper, 13 are Broad Absorption Line quasars. Five
quasars, including one object at a redshift of 5.11, have 20 cm peak flux
densities greater than 1 mJy. Two of the quasars, both at z=4.5, have very weak
emission lines; one of these objects is a radio source. Nineteen of the
newly-discovered objects have redshifts above 4.6, and the maximum redshift is
z=5.41; among objects reported to date, the latter is the third highest
redshift AGN, and penultimate in redshift among luminous quasars.Comment: 34 pages, 7 figures, submitted to A
PYTHIA 6.4 Physics and Manual
The PYTHIA program can be used to generate high-energy-physics `events', i.e.
sets of outgoing particles produced in the interactions between two incoming
particles. The objective is to provide as accurate as possible a representation
of event properties in a wide range of reactions, within and beyond the
Standard Model, with emphasis on those where strong interactions play a role,
directly or indirectly, and therefore multihadronic final states are produced.
The physics is then not understood well enough to give an exact description;
instead the program has to be based on a combination of analytical results and
various QCD-based models. This physics input is summarized here, for areas such
as hard subprocesses, initial- and final-state parton showers, underlying
events and beam remnants, fragmentation and decays, and much more. Furthermore,
extensive information is provided on all program elements: subroutines and
functions, switches and parameters, and particle and process data. This should
allow the user to tailor the generation task to the topics of interest.Comment: 576 pages, no figures, uses JHEP3.cls. The code and further
information may be found on the PYTHIA web page:
http://www.thep.lu.se/~torbjorn/Pythia.html Changes in version 2: Mistakenly
deleted section heading for "Physics Processes" reinserted, affecting section
numbering. Minor updates to take into account referee comments and new colour
reconnection option
- …